feat(demo): run real CI on the github rung - #604
Merged
Conversation
mnoah1
approved these changes
Aug 17, 2026
behinddwalls
force-pushed
the
preetam/github-actions-runner
branch
from
August 18, 2026 15:25
6a563c2 to
a6eedd2
Compare
behinddwalls
force-pushed
the
preetam/github-actions-runner
branch
from
August 18, 2026 16:28
a6eedd2 to
886e79b
Compare
behinddwalls
force-pushed
the
preetam/github-actions-runner
branch
from
August 18, 2026 16:44
886e79b to
060ae51
Compare
behinddwalls
force-pushed
the
preetam/github-actions-runner
branch
from
August 18, 2026 16:55
060ae51 to
a24ade0
Compare
behinddwalls
force-pushed
the
preetam/github-actions-runner
branch
from
August 18, 2026 17:15
a24ade0 to
c2df7d4
Compare
## Summary ### Why? The GitHub rung was described as the one where everything is real, and it was not. It took real pull requests from a real repository and pushed real commits back — but the build in the middle was the fake runner, passing instantly. `landed` there meant the queue had run its pipeline, not that anything had been built. That is a bad thing to be unclear about, because it is the one rung a reader is most likely to trust. Watching three pull requests merge in twenty seconds looks like a submit queue validating and landing them; nothing in the output says the validation was a stub. It was also the last thing separating the demo from the product. Speculation exists to test the *combination* — a batch on top of the in-flight changes assumed to land before it — and a fake runner exercises the trigger-and-poll loop while proving nothing about the combination it was built to check. The switch was already documented as an opt-in, three manual steps in a section further down. Nobody who has just watched the demo will go and do them, which meant the interesting half of the feature was reachable in principle and unused in practice. ### What? `demo/provider/github/profiles.yaml` selects the `githubactions` runner for `demo-queue` instead of inheriting the fake one. That is the whole change in behaviour: the queue now dispatches a `workflow_dispatch` per speculative batch, polls the resulting check, and decides merge or eject on the answer. The defaults are untouched, so a queue not listed still cannot reach GitHub by accident. `fake` and `git` are untouched: their builds stay instant, which is what keeps the first two rungs free and fast. The docs stop describing this as opt-in. "Using real CI" now covers what the *repository* must provide rather than what to switch on — a dispatchable workflow accepting the four `sq_*` inputs, and, the part that decides whether any of it means anything, a workflow that actually applies both input sets. A workflow that checks out the pull request and builds it is green about the wrong thing. Two consequences are stated where a reader meets them rather than buried: a land now takes as long as the workflow does, and every speculation path dispatched is a billable run — which makes the queue's build budget a spending control as much as a CI one. The ladder table gains a **Building it** column. It previously described where changes come from and what landing does, and said nothing about the build, which is how the fake runner stayed invisible in the summary a reader skims. The same paragraph now also says plainly that `fake` and `git` fake the *change provider* too, so `sq-files=` is how the analyzer sees paths there — true before this change and undocumented. Token permissions move Actions from "only if you switch the build runner" to required, and the classic-PAT line from "`repo`, plus `workflow` if…" to both. ## Test Plan - ✅ ran it: `PROVIDER=github make demo-requests COUNT=2` against a scratch repository opened pull requests 525 and 526 and landed both - ✅ the dispatches are real and are the queue's, confirmed from the Actions API rather than from the queue's own logs: two new `workflow_dispatch` runs at 18:53:26, both `success`, titled `SQ - demo-queue - speculative batch` — the workflow's own `run-name`, built from the `sq_queue` input the runner sent - ✅ before this change the same repository showed no `workflow_dispatch` run since 2026-08-10 and zero check-runs on a landed head, which is what established the gap in the first place - ✅ `make fmt`, `make lint` Not verified: cancellation. The runner cancels a build whose speculation path is abandoned, but the workflow used here finishes in seconds, so a run is almost always complete before there is anything to cancel — the same reason abandoned-path builds show up as post-terminal events in a fake run. It needs a slow workflow to exercise honestly. A note for anyone pointing this at their own repository: `ci.yml` and `behinddwalls/sq-demo` are marked in the file the same way `merge.yaml`'s remote is. A workflow triggered only by `pull_request` cannot be dispatched at all, and GitHub rejects the call rather than falling back.
behinddwalls
force-pushed
the
preetam/github-actions-runner
branch
from
August 18, 2026 17:23
c2df7d4 to
4f07326
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
The GitHub rung was described as the one where everything is real, and it was not. It took real pull requests from a real repository and pushed real commits back — but the build in the middle was the fake runner, passing instantly.
landedthere meant the queue had run its pipeline, not that anything had been built.That is a bad thing to be unclear about, because it is the one rung a reader is most likely to trust. Watching three pull requests merge in twenty seconds looks like a submit queue validating and landing them; nothing in the output says the validation was a stub.
It was also the last thing separating the demo from the product. Speculation exists to test the combination — a batch on top of the in-flight changes assumed to land before it — and a fake runner exercises the trigger-and-poll loop while proving nothing about the combination it was built to check.
The switch was already documented as an opt-in, three manual steps in a section further down. Nobody who has just watched the demo will go and do them, which meant the interesting half of the feature was reachable in principle and unused in practice.
What?
demo/provider/github/profiles.yamlselects thegithubactionsrunner fordemo-queueinstead of inheriting the fake one. That is the whole change in behaviour: the queue now dispatches aworkflow_dispatchper speculative batch, polls the resulting check, and decides merge or eject on the answer.The defaults are untouched, so a queue not listed still cannot reach GitHub by accident.
fakeandgitare untouched: their builds stay instant, which is what keeps the first two rungs free and fast.The docs stop describing this as opt-in. "Using real CI" now covers what the repository must provide rather than what to switch on — a dispatchable workflow accepting the four
sq_*inputs, and, the part that decides whether any of it means anything, a workflow that actually applies both input sets. A workflow that checks out the pull request and builds it is green about the wrong thing.Two consequences are stated where a reader meets them rather than buried: a land now takes as long as the workflow does, and every speculation path dispatched is a billable run — which makes the queue's build budget a spending control as much as a CI one.
The ladder table gains a Building it column. It previously described where changes come from and what landing does, and said nothing about the build, which is how the fake runner stayed invisible in the summary a reader skims. The same paragraph now also says plainly that
fakeandgitfake the change provider too, sosq-files=is how the analyzer sees paths there — true before this change and undocumented.Token permissions move Actions from "only if you switch the build runner" to required, and the classic-PAT line from "
repo, plusworkflowif…" to both.Test Plan
PROVIDER=github make demo-requests COUNT=2against a scratch repository opened pull requests 525 and 526 and landed bothworkflow_dispatchruns at 18:53:26, bothsuccess, titledSQ - demo-queue - speculative batch— the workflow's ownrun-name, built from thesq_queueinput the runner sentworkflow_dispatchrun since 2026-08-10 and zero check-runs on a landed head, which is what established the gap in the first placemake fmt,make lintNot verified: cancellation. The runner cancels a build whose speculation path is abandoned, but the workflow used here finishes in seconds, so a run is almost always complete before there is anything to cancel — the same reason abandoned-path builds show up as post-terminal events in a fake run. It needs a slow workflow to exercise honestly.
A note for anyone pointing this at their own repository:
ci.ymlandbehinddwalls/sq-demoare marked in the file the same waymerge.yaml's remote is. A workflow triggered only bypull_requestcannot be dispatched at all, and GitHub rejects the call rather than falling back.